Learning Features of Music from Scratch

We introduce a new large-scale music dataset, MusicNet, to serve as a source of supervision and evaluation of machine learning methods for music research. MusicNet consists of hundreds of freely-licensed classical music recordings by 10 composers, written for 11 instruments, together with instrument/note annotations resulting in over 1 million temporal labels on 34 hours of chamber music performances under various studio and microphone conditions. We define a multi-label classification task to predict notes in musical recordings, along with an evaluation protocol. We benchmark several machine learning architectures for this task: i) learning from “hand-crafted” spectrogram features; ii) end-to-end learning with a neural net; iii) end-to-end learning with a convolutional neural net. We show that several end-to-end learning proposals outperform approaches based on learning from hand-crafted audio features.